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DETAILED ACTION 
Response to Amendment 

1 . In response to the Office Action mailed August 16, 2004, applicant submitted an 
amendment filed on November 18, 2004, in which the applicant traversed and 
requested reconsideration with respect to claims 1 and 15. 

Response to Arguments 

2. Applicants argue that Van Schyndel fails to disclose a display system associated 
with a user, and also fails to disclose any visual indication on the display system 
indicating who is currently speaking during an event. However, the examiner disagrees. 
Van Schyndel discloses that the display of visual indicators is that of the visual and 
audio showing the individual who is speaking when camera 30 is pointing at that 
individual and transmission of this data (figure 5, element 105 with column 7, lines 35- 
46 and column 9, lines 22-36). 

Applicants also argue that there is no discussion in Van Schyndel of a 
determination of whether the current speaker is within a field of "audible perception" of 
the user. However, Van Schyndel discloses that the directional microphone is moved in 
the direction of imminent speakers only if they are within the audible perception of the 
user as determined by the system. When the user is teleconferencing, the user's 
audible field of perception, in the broadest sense, is defined by what he can hear 
through the teleconferencing system and his field of view is defined by the position and 
the field of view of the camera in the conference room (figure 5, element 50 with column 
9, lines 4-6 and lines 23-45). 
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Applicants also argues that the Van Schyndel reference teaches away fro the 
present invention, which recites a display system associated with a user, which provides 
the user with one or more visual indications. One of the visual indicators is provided to 
the user when the current speaker is not within the field of view of the user. This allows 
the user to change his/her own field of view. Thus, the applicant's argue that the 
present invention uses a human user as an essential aspect. However, the passage 
cited states that this eliminates the need for a human camera operator in an auditorium 
or conference hall environment. The applicant's argue something not claimed, which is 
the need of a human camera operator. In response to applicant's argument that the 
references fail to show certain features of applicant's invention, it is noted that the 
features upon which applicant relies (i.e., human camera operator) are not recited in the 
rejected claim(s). Although the claims are interpreted in light of the specification, 
limitations from the specification are not read into the claims. See In re Van Geuns, 988 
F .2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). 

Applicant's argue regarding claims 2 and 16-17 that both the present invention 
and Budd are both assigned to International Business Machines. Applicant's 
arguments, see page 5, filed November 18, 2004, with respect to the rejection(s)of 
claim(s) 2 and 16-17 under 35 USC 103 have been fully considered and are persuasive. 
Therefore, the rejection has been withdrawn. However, upon further consideration, a 
new ground(s) of rejection is made in view of Lemelson et al. 
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In response to applicant's argument that there is no suggestion to combine the 
references, the examiner recognizes that obviousness can only be established by 
combining or modifying the teachings of the prior art to produce the claimed invention 
where there is some teaching, suggestion, or motivation to do so found either in the 
references themselves or in the knowledge generally available to one of ordinary skill in 
the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988) and in re 
Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992). In this case, applicants argue 
that claims 6-8, 9-1 1 and 18-20 does not establish a prima facie case of obviousness. 

Regarding claim 6, Potts teaches that determining the location of the speaker 
based on audio information helps to improve the speaker detection, since the combined 
use of both audio and video detection modules is used to identify the current speaker 
and would improve the overall reliability of the system (Potts; column 4, lines 25-29). 

Regarding claims 7-8, Van Schyndel teaches capturing directional data 
associated with display system and positional data associated with the user (step 245, 
FIG. 2). Since the field of view of the user of the teleconferencing system in Van 
Schyndel is always the view through the center camera, the position of the current talker 
is determined as an offset from the central camera position, and the microphones are 
moved accordingly (FIG. 4a and Col. 7, lines 50-55), in which motivation to combine 
was not provided since the primary reference was the only reference used. 

Regarding claims 9-1 1 and 18-20, Hein teaches using a colored frame or shared 
cursor (marker) around the speaker's image, in order to identify him to the viewer where 
the frame can change color or change background (column 7, lines 8-1 1 ). As it is well 
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known in the art, this indicator would point out the current speaker from a group of 
people (Hein; column 3, lines 7-9) 

Therefore, applicant's arguments filed November 18, 2004 have been fully 
considered but they are not persuasive. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1, 3, 4 and 15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Van Schyndel (USPN 5,940,118). 

As per claims 1 and 15, Van Schyndel discloses: 

a method of providing a user (110, FIG. 5) with one or more visual indications, in 
accordance with a display system associated with the user, of who is currently speaking 
during an event in which the user is engaged, the event including one or more other 
individuals, the method comprising the steps of: 

• identifying the location of the individual who is currently speaking during the 
event (20, FIG. 5 with column 3, lines 50-54 and column 7, lines 35-46); 
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• determining whether the individual identified as the current speaker is within a 
field of audible perception of the user (50, FIG. 5 and Col. 9, lines 4-6 with lines 23-45). 
The directional microphone is moved in the direction of imminent speakers only if they 
are not within the audible perception of the user as determined by the system. When the 
user is teleconferencing, the user's audible field of perception, in the broadest sense, is 
defined by what he can hear through the teleconferencing system and his field of view is 
defined by the position and the field of view of the camera in the conference room); 

• displaying a first visual indicator to the user, in accordance with the display 
system, in association with the individual identified as the current speaker when the 
individual is within the field of view of the user (105, FIG. 5 and Col. 7, lines 35-46, Col. 
9, lines 22-36, i.e., the video and audio showing the individual who is speaking when 
camera 30 is pointing at that individual and transmission of this data to 105); 

Van Schyndel (preferred embodiment) does not disclose (1) the use of 
determining whether the individual identified as the current speaker is within a field of 
view of the user and (2) displaying a second visual indicator to the user, in accordance 
with the display system, when the individual identified as the current speaker is not 
within the field of view of the user. 

However, Van Schyndel (background art) does disclose that it is well-known to 
direct a camera at a determined sound source to provide coordinated video/audio and 
eliminate the need for human operator (See Col. 2, lines 27-34). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Van Schyndel's method in order to determine 
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whether the current speaker is within the field of view of the user as well as the field of 
audible perception and then direct the camera toward the speaker for display to the user 
in Van Schyndel (i.e. displaying a second visual indicator, via the monitor 105 that 
would show redirecting of the camera) as suggested by Van Schyndel as being well- 
known, the motivation being to provide coordinated video/audio and eliminate the need 
for a human operator. 

As per claim 3, Van Schyndel discloses: 

• capturing one or more video images of the one or more individuals participating 
in the event (Col. 7, lines 35-36); 

• analyzing the one or more captured video images to determine which individual 
has one or more facial features indicative of speech (Col.7, lines 39-41); 

• designating the individual with the one or more facial features indicative of 
speech as the current speaker ('dcurrent talker", Col. 7, lines 42-43); 

• and determining the location of the individual designated as the current speaker 
(Col. 7, lines 45-50). 

As per claim 4, Van Schyndel teaches capturing one or more video images of the field 
of the user (step 245, FIG. 2). Since the field of view of the user of the teleconferencing 
system in Van Schyndel is always the view through the center camera, the position of 
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the current talker is determined as an offset from the central camera position, and the 
microphones are moved accordingly. (FIG. 4a and Col. 7, lines 50-55) 

5. Claims 2 and 16-17 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Van Schyndel as applied to claim 1 above, and further in view of Lemelson et al. 
(USPN 6,351,273), hereinafter referenced as Lemelson. 

As per claims 2 and 16-17, Van Schyndel discloses: 

a stationary display system (105, FIG. 5), but does not disclose the use of display 
system worn by the user. 

Lemelson teaches head-mounted display system (FIG. 1 , elements 32, 18 and 

22). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Van Schyndel as taught by Lemelson, in order to 
provide a hands free environment for the user (column 8, lines 49-65) 
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6. Claims 6-8 are rejected under 35 U.S.C. 103(a) as being unpatentable over Van 
Schyndel as applied to claim 1 above, and further in view of Potts et al. (USPN 
6,593,956), hereinafter referenced as Potts. 

As per claim 6, Van Schyndel discloses: 

• capturing audio data of the one or more individuals participating in the event 
(inherent during the use of microphones, 60, FIG. 5); 

• analyzing the audio data to determine which individual is uttering sound 
indicative of speech (steering microphone towards the direction of incoming 
sound, Col. 5, lines 5-8 and step 335, FIG. 3); and 

• designating the individual uttering sound that is indicative of speech as the 
current speaker (determining whether the "imminent talker" is currently speaking, Col. 9, 
lines 10-19). 

However, Van Schyndel does not disclose the use of determining the location of 
the individual designated as the current speaker using audio information. 

Potts teaches determining the location of the speaker based on audio information 
(114, FIG. 4 and Col. 17, lines 35-55) 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Van Schyndel as taught by Potts, in order to 
improve the speaker detection, since the combined use of both audio and video 
detection modules to identify the current speaker would improve the overall reliability of 
the system (Col. 4, lines 25-29). 
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As per claims 7-8, Van Schyndel teaches capturing directional data associated with 
display system and positional data associated with the user (step 245, FIG. 2). 
Since the field of view of the user of the teleconferencing system in Van Schyndel is 
always the view through the center camera, the position of the current talker is 
determined as an offset from the central camera position, and the microphones are 
moved accordingly (FIG. 4a and Col. 7, lines 50-55). 

7. Claims 9-11 and 18-20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Van Schyndel as applied to claim 1 above, and further in view of 
Hein et al. (USPN 6,466,250), hereinafter referenced as Hein. 

As per claims 9-11 and 18-20, Van Schyndel discloses: 

a method for providing a user with one or more visual indications of who is 
currently speaking, but does not disclose that a "first visual indicator comprises a 
marker displayed in proximity to a representation of the individual identified as the 
current speaker on the display system." 

Hein teaches using a colored frame or a shared cursor (marker) around the 
speaker's image in order to identify him to the viewer where the frame can change color 
or change background (Col. 7, lines 8-11). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Van Schyndel et al. as taught by Hein, in order 
to identify the current speaker located in the field of view of the user, because, as it is 
well-known in the art, the indicator would point out the current speaker from a group of 
people (Hein; Col. 3, lines 7-9) 

8. Claims 13-14 and 22 are rejected under 35 US.C. 103(a) as being unpatentable 
over Van Schyndel as applied to claim 1 above, and further in view of Butnaru et al. 
(USPN 6,240,392), hereinafter referenced as Butnaru. 

As per claims 13-14 and 22, Van Schyndel discloses: 

a method for providing a user with one or more visual indications of who is 
currently speaking, but does not disclose obtaining textual transcription of audio content 
through either human stenography or speech recognition, and displaying textual 
transcription on the display system. 

Butnaru discloses the system that is capable of recognizing the speech using 
speech recognizer (55, FIG. 3) and displaying the text content to the user via a display 
system (Col. 5, lines 49-57). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Van Schyndel as taught by Butnaru, to enable 
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the deaf people using the system to padicipate in the videoconferencing or other forms 
of telecommunications (Col. 1 , line 63 - Col. 2, line 8). 

Allowable Subject Matter 

9. Claims 5, 12 and 21, are objected to as being dependent upon a rejected base 
claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

1 0. Independent claim 23 is allowed. 

The following is a statement of reasons for the indication of allowable subject matter: 

prior art does not teach or suggest displaying the second indicator to a user, directing 
him to turn his head in the direction of the speaker (claims 12, 21) and displaying the 
first visual indicator in such a way that the images of the current speakers and field of 
view of the user are combined (claim 5). Claim 23 combines all of these individual 
limitations, and hence is also allowable. 

Dependent claims 24-28 are allowed as dependent on independent claim 23 and 
further limiting its scope. 
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Conclusion 

1 1 . The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

> Dunn et al. (USPN 5,916,302) discloses a multimedia conferencing using parallel 
networks. 

> Marshall (USPN 5,473,726) discloses audio and amplitude modulated phot data 
collection for speech recognition. 

> Addeo et al. (USPN 5,335,01 1) discloses a sound localization system for 
teleconferencing using self-steering microphone arrays. 

> Greene, JR. et al. (USPN 6,377,925) discloses electronic translator assisting 
communications. 

> Sawaga et al. (USPN 5,734,923) discloses an apparatus for interactively editing 
and outputting sign language information using graphical user interface. 

12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jakieda R Jackson whose telephone number is 

571 .272.7619. The examiner can normally be reached on Monday through Friday from 
7:30 a.m. to 5:00p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David Ometz can be reached on 571 .272.7593. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



JRJ 

April 29, 2005 




OAVID L. OMETZ 
PRIMARY EXAMINER 



